Mirroring network traffic of virtual networks at a service provider network

ABSTRACT

Techniques are described that enable users to configure the mirroring of network traffic sent to or received by computing resources associated with a virtual network of computing resources at a service provider network. The mirrored network traffic can be used for many different purposes including, for example, network traffic content inspection, forensic and threat analysis, network troubleshooting, data loss prevention, and the like. Users can configure such network traffic mirroring without the need to manually install and manage network capture agents or other such processes on each computing resource for which network traffic mirroring is desired. Users can cause mirrored network traffic to be stored at a storage service in the form of packet capture (or “pcap”) files, which can be used by any number of available out-of-band security and monitoring appliances including other user-specific monitoring tools and/or other services of the service provider network.

BACKGROUND

Service provider networks provide users with the ability to utilize avariety of types of computing-related resources such as computeresources (for example, executing virtual machine (VM) instances orcontainers), data/storage resources, network-related resources,application resources, access policies or roles, identity policies orroles, machine images, routers and other data processing resources, andso forth. These and other computing resources can be provided asservices, such as a hardware virtualization service that can executecompute instances, a storage service that can store data objects, andthe like.

Users of some such service provider networks can create virtual networksin which users can provision logically isolated collections of computingresources within a service provider network. Users can customize thenetwork configuration for such virtual networks, including the creationof public-facing and private-facing subnets, configuration of securitygroups and network access control lists, and so forth. The ability tomanage and analyze the actual network traffic traversing such virtualnetworks for however presents several challenges.

BRIEF DESCRIPTION OF DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 is a diagram illustrating an environment for mirroring networktraffic associated with virtual networks of a service provider networkaccording to some embodiments.

FIG. 2 is a diagram illustrating components of a compute instance fleetused to process network traffic mirrored from users' virtual networks ata service provider network according to some embodiments.

FIG. 3 illustrates an example of a compute instance fleet convertingmirrored network traffic to packet capture data according to someembodiments.

FIG. 4 is a diagram illustrating the use of various ancillary servicesof a service provider network to process and analyze packet capture datastored at a storage service according to some embodiments.

FIG. 5 is a flow diagram illustrating operations of a method forenabling users to mirror network traffic associated with a virtualprivate network at a service provider network according to someembodiments.

FIG. 6 illustrates an example service provider network environmentaccording to some embodiments.

FIG. 7 is a block diagram of an example provider network that provides astorage service and a hardware virtualization service to customersaccording to some embodiments.

FIG. 8 illustrates an example provider network that provides virtualnetworks on the provider network to at least some customers according tosome embodiments.

FIG. 9 illustrates subnets and security groups in an example virtualnetwork on a provider network according to some embodiments.

FIG. 10 is a block diagram illustrating an example computer system thatmay be used in some embodiments.

DETAILED DESCRIPTION

Various embodiments of methods, apparatus, systems, and non-transitorycomputer-readable storage media are described that enable users toconfigure the mirroring of network traffic sent to or received bycomputing resources associated with a virtual network of computingresources at a service provider network. The ability to gain visibilityinto such network traffic associated with users' virtual networks can bebeneficial for any of several desired purposes including, for example,network traffic content inspection, forensic and threat analysis,network troubleshooting, data loss prevention, and so forth. Accordingto embodiments described herein, users can configure such networktraffic mirroring relative to their virtual networks, including themirroring of full payload data, without the need to manually install andmanage network capture agents or other such processes on each computingresource for which network traffic mirroring is desired. In someembodiments, the mirrored network traffic is sent to a compute instancefleet that is designed to process the network traffic and to forward theprocessed data to one or more user-specified destinations, which caninclude various streaming and storage services. As one example, theprocessed network traffic can be stored at a storage service of aservice provider network in the form of packet capture (or “pcap”)files, which can be used by any number of other out-of-band securityservices, monitoring appliances, or any other services or applications.

A virtual network generally enables users to provision logicallyisolated collections of computing resources at a service providernetwork and to manage various related capabilities such as securityservices, firewalls, and intrusion detection systems. A virtual networkmay in some ways closely resemble a traditional network operated, forexample, in a user's own on-premises data center. A virtual network canbe logically isolated from other virtual networks associated with a sameuser account as well as those virtual networks associated with otheruser accounts of the service provider network.

In traditional data centers, network administrators can install varioussoftware agents or hardware devices to capture network traffictransmitted by devices in a network to detect various types of networkand security issues. Embodiments described herein provide these andother types of network traffic monitoring features for computingresources of users' virtual networks at a service provider network.Computing devices that implement computing resources within a user'svirtual network can be configured to create a mirrored copy of selectnetwork traffic and to send the mirrored network traffic to auser-specified destination. In some examples, users can cause themirrored traffic to be stored at storage resources provided by theservice provider network and at which users can retrospectively analyzethe mirrored network traffic. Users could optionally run individualagents on their compute instances in a virtual network to perform basicnetwork traffic mirroring; however, the configuration of agents atindividual compute instances fails to scale well, particularly as users'virtual networks may grow to include hundreds, thousands, or even morecompute instances.

Embodiments described herein enable users to easily configure theircompute instances and other service provider network-managed resourcessuch as load balancers, internet gateways, network address translation(NAT) gateways, and so forth, to mirror network traffic according touser configurations. As indicated above, for example, network trafficcan be mirrored to a compute instance fleet that processes the data andcan route the mirrored network traffic to one or more streamingservices, storage services, or other user-specified destinations forfurther processing and analysis. The processing by the compute instancefleet can include the addition of various types of metadata and/orconversion of the network traffic into packet capture files or into anyother representation of network traffic. The stored packet capture dataor data streamed from the compute instance fleet can be used by multipletools and algorithms (including, for example, machine learning (ML)based tools) to detect potential network or security related issues.Among other benefits, the ability to configure such traffic mirroringenables users to easily monitor network traffic associated with theirvirtual networks at service provider networks and provides a consistentway to access, categorize, and consolidate the delivery of networktraffic to any type of destination, including out-of-band security andmonitoring tools or other services available at a service providernetwork.

FIG. 1 is a block diagram illustrating an environment that enables usersto configure the mirroring of network traffic sent to or received bycomputing resources associated with a virtual network of computingresources at a service provider network. In some embodiments, a hardwarevirtualization service 106 and a storage/streaming service(s) 104, amongany number of other possible services, operate as part of a serviceprovider network 100 and each comprises one or more software modulesexecuted by one or more electronic devices at one or more data centersand geographic locations. A user 110 using one or more electronicdevice(s) 112 (which may be part of or separate from the serviceprovider network 100) can interact with the various services of theservice provider network 100 via one or more networks, such as theinternet.

A service provider network 100 provides users with the ability toutilize one or more of a variety of types of computing-related resourcessuch as compute resources (for example, executing virtual machine (VM)instances and/or containers, executing batch jobs, executing codewithout provisioning servers), data/storage resources (for example,object storage, block-level storage, data archival storage, databasesand database tables, and so forth), network-related resources (forexample, configuring virtual networks including groups of computeresources, content delivery networks (CDNs), Domain Name Service (DNS)),application resources (for example, databases, applicationbuild/deployment services), access policies or roles, identity policiesor roles, machine images, routers and other data processing resources,and so forth. These and other computing resources may be provided asservices, such as a hardware virtualization service that can executecompute instances, a storage service that can store data objects, and soforth. The users (or “customers”) of provider networks 100 may utilizeone or more user accounts that are associated with a customer account,though these terms may be used somewhat interchangeably depending uponthe context of use. Users may interact with a provider network 100across one or more intermediate networks 114 (for example, the internet)via one or more interface(s) 116, such as through use of applicationprogramming interface (API) calls, via a console implemented as awebsite or application, etc. The interface(s) 116 may be part of, orserve as a front-end to, a control plane 118 of the provider network 100that includes “backend” services supporting and enabling the servicesthat may be more directly offered to customers.

To provide these and other computing resource services, providernetworks 100 often rely upon virtualization techniques. For example,virtualization technologies may be used to provide users the ability tocontrol or utilize compute instances (for example, a VM using a guestoperating system (O/S) that operates using a hypervisor that may or maynot further operate on top of an underlying host O/S, a container thatmay or may not operate in a VM, an instance that can execute on “baremetal” hardware without an underlying hypervisor), where one or multiplecompute instances can be implemented using a single electronic device.Thus, a user may directly utilize a compute instance hosted by theprovider network to perform a variety of computing tasks, or mayindirectly utilize a compute instance by submitting code to be executedby the provider network, which in turn utilizes a compute instance toexecute the code (typically without the user having any control of orknowledge of the underlying compute instance(s) involved).

As shown in FIG. 1 , a user has configured a virtual network 102 at theservice provider network 100. As indicated above, a virtual network 102represents a network in which a user of the service provider network canprovision a logically isolated set of computing resources. To establisha virtual network 102, a user can allocate one or more computingresources (for example, VMs 120A-120C) to the virtual network 102. Insome embodiments, a virtual network 102 can include any number of othertypes of resources including containers, block storage, databases,network-related resources, and so forth. A range of public IP addressescan also be allocated to the virtual network 102 and one or morenetworking nodes (for example, routers, switches, and so forth) of theprovider network 100 can be allocated to the virtual network 102. Insome embodiments, a virtual network 102 can include one or more of aprivate gateway 124 and include a public gateway 126, which enablescomputing resources within the virtual network 102 to communicatedirectly with entities via intermediate networks 114, and vice versa,instead of or in addition to via a private communications channel.

In an embodiment, a virtual network 102 can be optionally subdividedinto two or more subnetworks, or subnets, 130A, 130B. For example, inimplementations that include both a private gateway 124 and a publicgateway 126, a virtual network 102 can be subdivided into a subnet 130Athat includes resources (VM 120A, in this example) reachable throughprivate gateway 124, and a subnet 130B that includes resources (VM 120B,in this example) reachable through public gateway 126.

In an embodiment, a user can assign allocated public IP addresses toresources in a virtual network 102. A network entity on an intermediatenetwork 114 can then send traffic to a public IP address published bythe user; the traffic is routed, by the service provider network 100, tothe associated compute instance. Return traffic from the computeinstance is routed, by the service provider network 100, back to thenetwork entity over intermediate network 114. Note that routing trafficbetween a compute instance and the network entity may require networkaddress translation to translate between the public IP address and thelocal IP address of the compute instance.

In an embodiment, each computing resource of a user's virtual network102 can be associated with one or more virtual network interfaces (forexample, the VMs 120A running at computing device(s) 132A hosting VMs120A can be associated with one or more virtual network interfaces 134A,the VMs 120B running at computing device(s) 132B hosting VMs 120B can beassociated with one or more virtual network interfaces 134B, and the VMs120C running at computing device(s) 132C hosting VMs 120C can beassociated with one or more virtual network interfaces 134C). A virtualnetwork interface is a logical networking component of a virtual network102 that represents a virtual network card. These virtual networkinterfaces can be created in association with a user's account andattached to VMs in a user's virtual network 102. Virtual networkinterfaces can also be detached from compute instances and attached toother compute instances, where the attributes of a network interfacefollow it as it's attached or detached from various instances. In someembodiments, users can also modify the attributes of a virtual networkinterface, including changing its security groups and managing IPaddresses associated with the interface.

In some embodiments, virtual network interfaces can be implemented atleast in part at one or more offload cards (which include one or moreprocessors, and possibly include one or more physical networkinterfaces) that are connected to computing devices 132A-132C using anI/O interface (for example, a bus implementing a version of thePeripheral Component Interconnect-Express (PCI-E) standard, or anotherinterconnect such as a QuickPath interconnect (QPI) or UltraPathinterconnect (UPI)). For example, in some embodiments the computingdevices 132A-132C host compute instances (for example, VMs 120A-120C),and the one or more offload cards execute a virtualization manager andother software that can manage compute instances and associated virtualnetwork interfaces at the host electronic device. As an example, in someembodiments the offload card(s) can perform compute instance managementoperations such as pausing and un-pausing compute instances, launchingand terminating compute instances, performing memory transfer/copyingoperations, routing network traffic, and so forth. In some embodiments,these management operations are performed by the offload card(s) incoordination with a hypervisor (for example, upon a request from ahypervisor) that is executed by other processors of a computing device.However, in some embodiments, the virtualization manager implemented bythe offload card(s) can accommodate requests from other entities (forexample, from compute instances themselves) and may not coordinate withor service any separate hypervisor.

According to embodiments described herein, one configurable attribute ofa virtual network interface is whether to enable mirroring of networktraffic traversing the network interface (that is, network trafficeither received or being sent by the network interface). As indicatedabove, it is sometimes desirable for users to enable mirroring ofnetwork traffic associated with computing resources of a virtual networkfor any of several reasons including, for example, content inspection,forensic and threat analysis, network troubleshooting, and so forth. Atthe circle labeled “1” in FIG. 1 , a hardware virtualization service, orany other service used to manage virtual networks at the serviceprovider network, receives a request to enable a traffic mirroringsession at one or more virtual network interfaces associated with avirtual network of a service provider network. In some embodiments, therequest can be received via an application programming interface (API),command-line interface (CLI), web-based console, or any other interfaceprovided by the service. In an embodiment, a request to create a trafficmirroring session identifies one or more source virtual networkinterfaces (that is, where to perform the requested network trafficmirroring), one or more destinations (that is, where the mirrorednetwork traffic is to be sent), optional filters identifying what typesof network traffic is to be mirrored (for example, only ingress oregress traffic, only traffic associated with particular protocols, andso forth), among other possible parameters.

In one embodiment, the source(s) identified in a network trafficmirroring request can include one or more of: one or more identifiedvirtual network interfaces (for example, identified by a user based onidentifiers associated with the virtual network interfaces), one or moreidentified subnets of a virtual network, an entire virtual network,public or private gateways, based on metadata identifying a set ofvirtual network interfaces (for example, based on one or more “tags”associated with virtual network interfaces in a user's account), or anycombinations thereof. The identified virtual network interfaces, forexample, can be associated with various compute instances (for example,any of VMs 120A-120C), public or private gateways, load balancers, NATgateways, or any other computing resources within a virtual network 102.In some embodiments, users can request to mirror network traffictraversing particular network paths, for example, only network traffictraversing a public gateway or private gateway of a virtual network.

In an embodiment, a received request can alternatively be a request tostop, pause, restart, or delete a previously created network trafficmirroring session at one or more virtual network interfaces. In someembodiments, a request can instead be a request to modify an existingnetwork traffic mirroring session, for example, to modify the existingsession to include or remove previously identified mirroring sources, tochange, add, or remove a destination for the mirrored network traffic,or to modify any other attribute associated with a previously creatednetwork traffic mirroring session. In these examples, a service of theservice provider network receiving such a request can generateadditional configuration data that is sent to the relevant computingdevices causing the computing devices to carry out the requestedoperation(s) relative to the identified traffic mirroring session(s).

In an embodiment, a request to create a network traffic mirroringsession can specify network traffic filtering options. For example,filtering options can be used to identify types of network traffic thata user desires to have mirrored, where only network traffic matching adefined filter is mirrored by the relevant computing devices. In otherexamples, filters can additionally or alternatively be used to excludeselected types of network traffic from being mirrored. Filtering rulesthat can be specified in a network traffic mirroring request caninclude, for example, a filter to mirror only inbound traffic receivedby one or more virtual network interfaces, or to mirror only outboundtraffic, or only network traffic associated with specified protocols(for example, only TCP, HTTP, or DNS traffic), only traffic associatedwith specified source ports, only traffic sent to specified destinationIP ranges, only traffic sent to specified destination port(s), and soforth. The following is an example of a command that can be used tocreate a network traffic mirroring session, including the specificationof mirroring source(s), destination(s), and various types of filters:“create-vpc-traffic-mirror -vpc-id <vpc-id> --mirror-source<mirror-source-ARN, IGW, VGW, Subnet, VPC> --mirror-destination<storage-bucket> [--traffic-direction <--ingress, egress, both>--snap-length <1-65535> -- Protocol < -- TCP, UDP, Any> --source-IP <IPaddress> -- source port < port number>> --destination-IP <IP address>--destination port < port number> --time-limit <seconds>--capture-frequency <seconds>].”

As another example use of filters, a user can specify a filter definingset of network addresses that the user desires their resources not toreceive network traffic from. When this example filter matches, onlypackets received from the defined set of network addresses may bemirrored. As another particular example, an employee of an organizationmight be associated with an IP address and an administrator desires toknow whenever that employee initiates an SSH connection into anyinstances of a particular subnet. That mirrored traffic can be sent to adestination, for example, that causes the administrator to receive anotification when such traffic is detected. As yet another example, afilter can be configured to mirror network traffic received from certainspecified geographic regions and a destination of the mirrored trafficcan be configured to update various security group rules in response.

In an embodiment, if a user requests to mirror network trafficassociated with an entire virtual network 102, for example, the customercan further specify whether the user desires to mirror onlyintra-network traffic, only inter-network traffic, or both. Similarly, arequest to mirror network traffic associated with one or more identifiedsubnets (for example, subnets 130A, 130B) can include an indication tomirror only intra-subnet traffic, only-inter network traffic, or both.

In one embodiment, in examples where the destination is a storageservice 104 used to store packet capture files created from the mirrorednetwork traffic, a user can specify one or more storage locations (forexample, one or more storage “buckets”) at the storage service 104 atwhich to store the created packet capture files. In other examples, adefault storage location can be created at the storage service 104 if auser does not explicitly identify a storage location as part of therequest.

In some embodiments, a network traffic mirroring request optionally canfurther specify a snapshot length indicating an amount of each networkpacket to capture (for example, the specified length can be from 1 byteto 65535 bytes), a time limit for the mirroring session (for example, adefault may be 600 seconds and a value of 0 can be specified to disablethe time limit), a capture frequency specifying the frequency of packetcapture (for example, a value of 60 may indicate that the networktraffic data is captured into separate files each having 60 seconds ofcaptured network packets, where any amount of time is possible), and amaximum capture file size (for example, a user can specify a maximumsize of each packet capture file before a new file is created), atriggering condition to initiate the mirroring session (for example, apattern or type of network traffic that, once identified at a virtualnetwork interface, causes the interface to start mirroring networktraffic).

At the circle labeled “2,” a hardware virtualization service processesthe request at least in part by identifying a set of computing devicesassociated with the virtual network interfaces identified by therequest, and generating configuration information used to configure theidentified computing devices. In one embodiment, the request isprocessed by a hardware virtualization service 106 or any other servicethat manages the virtual network 102. For example, if the networktraffic mirroring request received at circle “1” identifies one or moreparticular virtual network interfaces (for example, virtual networkinterface(s) 134A) associated with the virtual network 102, the hardwarevirtualization service 106 can identify computing devices (for example,computing device(s) 132A) hosting VMs to which the particular virtualnetwork interfaces are attached. The identification of the computingdevices can be based, for example, on mapping information maintained bythe hardware virtualization service 106 associating each virtual networkinterface with a compute instance and host computing device to which thevirtual network interface is attached.

If the user instead requests to mirror network traffic associated withan entire subnet or entire virtual network 102, the hardwarevirtualization service 106 determines which computing devices areassociated with virtual network interfaces in the virtual network towhich to send the configuration. For example, the hardwarevirtualization service 106 may store mapping information indicatingcomputing device-virtual network pairs and, based on the mapping, theservice can determine which computing devices manage interfaces for therelevant subnetwork(s) or the entire virtual network. In an embodiment,the hardware virtualization service 106 creates configurationinformation for each computing device with at least one relevant virtualnetwork interface attached, where the configuration information causesthe computing device to configure the relevant interfaces to mirrornetwork traffic according to the configuration. As indicated above, theconfiguration information can include user-specified configurationoptions including, for example, which source interfaces are to mirrornetwork traffic, which destination interfaces are to receive themirrored network traffic, properties of those destinations, optionalfiltering configurations, among other possible parameters.

At the circle labeled “3,” an example computing device receives theconfiguration information and configures one or more virtual networkinterfaces associated with the computing device to mirror networktraffic according to the configuration information generated at circle“2.” As indicated above, in some embodiments, the configuration of thenetwork traffic mirroring at a computing device can include configuringan offload card of the computing device to mirror network traffic sentto or received by the computing device according to the configuration.In other examples, the configuring can additionally or alternativelyinclude configuring one or more components of a hypervisor, VM, or otherapplications running on the computing device to perform the mirroring.Although the example in FIG. 1 shows a single computing device receivingconfiguration information, such configuration information may bereceived by and used to configure any number of computing devices in avirtual network 102 depending on the source(s) identified in the initialrequest.

At the circle labeled “4,” the computing device configured to mirrornetwork traffic sends or receives network traffic and creates a mirroredcopy of the network traffic according to the configuration received atcircle “3.” The network traffic generally can include any type ofnetwork traffic sent to or received by any application or processexecuting on the computing device, including network traffic sent to orreceived by any user compute instances or applications running at thecomputing device.

As indicated above, computing devices generally can be configured tomirror network traffic traversing one or more particular virtual networkinterfaces according to any defined filters and other user-specifiedconfigurations. In one embodiment, the mirroring of the network trafficgenerally includes creating a copy of the received/sent network packets,for example, at an offload card of the computing device. In someembodiments, creating the mirrored copy of the network traffic canfurther include creating an encapsulation layer that provides additionalinformation about the mirrored network traffic including, for example,identifiers of the associated virtual network interface, virtualnetwork, VM, user, network mirroring session, and so forth. As describedbelow, these identifiers can be used by various further processing stepsthat receive the mirrored network traffic.

At the circle labeled “5,” the computing device sends the mirrored copyof the network traffic, including any additional metadata, to one ormore destination network locations identified in the configurationinformation. According to one embodiment, the mirrored copy of thenetwork traffic is sent to a compute instance fleet 140 that optionallyperform one or more operations relative to the mirrored network trafficand route the mirrored traffic to a user-specified destination. Thedestination for the mirrored network traffic, for example, can be one ormore of: one or more user-specified network locations (for example, anetwork address corresponding to a virtual network interface attached toanother compute instance within the service provider network 100), oneor more services of the service provider network, one or more internalor external applications, or any other network-accessible location. Asshown in the example of FIG. 1 , the mirrored network traffic canoptionally be sent to one or more storage service(s) 104 or streamingservice(s) 142 of the service provider network.

At the circle labeled “6,” a compute instance fleet processes and routesthe mirrored network traffic to one or more destinations. For example,the compute instance fleet 140 can convert the mirrored network trafficdata to packet capture data 136 (including, for example, packet capturefiles 138A-138N) to be stored at one or more storage locations at astorage service 104 associated with the corresponding users of theservice provider network. In some embodiments, to enable the computeinstance fleet 140 to scale as an amount of mirrored network trafficincreases or decreases over time, the fleet of compute instances can beassociated with one or more auto scaling groups, network load balancers,and other possible components that distribute the received networktraffic over the available compute instances of the fleet. In oneembodiment, processing the mirrored network traffic can includeidentifying configuration information associated with the mirrorednetwork traffic (for example, based on one or more identifiers includedin the mirrored network traffic), optionally adding additional metadatato the mirrored network traffic, converting the network traffic to adifferent format (for example, converting the network packets to packetcapture data), compressing the data, encrypting the data, or performingany other operations on the received mirrored traffic.

As indicated above, the processing of the mirrored network traffic bythe compute instance fleet 140 can include converting the mirrorednetwork traffic into packet capture data. The packet capture datagenerally can include any file format that can be used by variousnetwork traffic analysis, network security, or other types ofapplications. For example, the packet capture data can be formatted asbinary data that can be processed by particular types of applications,or formatted in any of various plain-text formats such as CSV files orany other data format that can be used to represent network trafficinformation. Although the example of FIG. 1 shows the network trafficbeing sent to storage service(s) 104 and/or streaming service(s) 142, asdescribed below, the network traffic can be sent additionally oralternatively to any number of other types of services or applicationsincluding, for example, a network threat intelligence service, and soforth.

FIG. 2 is a diagram illustrating a compute instance fleet used toprocess network traffic mirrored from users' virtual networks at aservice provider network according to some embodiments. As shown in FIG.2 , a compute instance fleet 204 can be implemented to receive mirrorednetwork traffic 214 from any number of computing devices associated withany number of virtual networks 202A-202N within a service providernetwork 100. The compute instance fleet 204 can include any number ofcomputing devices 206A-206N each hosting any number of VMs 208A-208N,where the size of the fleet 204 can be scaled according to an amounttraffic being received from the virtual networks 202A-202N or based onamount of traffic expected to be received in the future.

In an embodiment, the compute instance fleet 204 processes mirrorednetwork traffic that it receives according to configuration informationassociated with the mirrored network traffic. For example, the computeinstance fleet can analyze the mirrored network traffic to identifyconfiguration information associated with the traffic. The configurationinformation can be identified based on one or more identifiers includedwith the mirrored network traffic such as, for example, user accountidentifiers, virtual network identifiers, virtual network interfaceidentifiers, IP addresses, and so forth. The configuration informationcan be obtained periodically by the compute instance fleet from anotherservice at which the network traffic mirroring is configured.

In an embodiment, the processing of the mirrored network trafficreceived by the compute instance fleet can include modifying and/orsupplementing the mirrored network traffic, and routing the mirrorednetwork traffic to one or more user-specified destinations (for example,a data storage service, a data streaming service, an application, and soforth). The modifying and/or supplementing of the mirrored networktraffic can include, for example, converting the mirrored networktraffic into another data format (into network packet files, forexample), segmenting the mirrored network traffic, aggregating two ormore separate network traffic streams, removing or modifying portions ofthe mirrored network traffic, adding metadata to the mirrored networktraffic, or any other operations specified by the configurationinformation. The compute instance fleet can also route the mirrorednetwork traffic, or the processed version thereof, to one or moredestinations as specified by the configuration information.

As shown in FIG. 2 , in one example, the mirrored network traffic 214 isprocessed by the compute instance fleet 204 and converted into packetcapture data, where the packet capture data is stored at separatestorage locations 216A-216N associated with separate user accounts atthe service provider network. For example, user 1 packet capture data210A is stored at a storage location 216A that is associated with afirst user account, while user 2 packet capture data 210B is stored at aseparate storage location 216B that is associated with a second useraccount, where each storage location can be associated with variousaccess rules to prevent unauthorized access to the data. The computeinstance fleet 204 can determine where to store received network trafficbased on identifiers associated with the network traffic (for example,user account identifiers, virtual network interface identifiers, virtualnetwork identifiers, and so forth).

As indicated above, in one embodiment, the packet capture files (forexample, packet capture files 212A-212N, 218, and 220A-220N) can includeadditional fields for metadata including information about the source ofthe mirrored network traffic. For example, the metadata included withthe packet capture files can include one or more of: a virtual networkidentifier (for example, from which of the virtual networks 202A-202Nwas the network traffic received), a user account identifier associatedwith the network traffic, a virtual network interface identifier, acompute instance identifier, a container/task identifier, tagsassociated with compute instances, and identifiers of filters used tocapture the traffic. In one embodiment, users can specify lifecycleconfigurations at a storage service 104 to manage the lifecycle ofstored packet capture data.

In some embodiments, the data stored at a storage service 104 can bequeried directly via the storage service. For example, the storageservice 104 can include functionality that enables users to view andquery packet capture data or other data formats stored at the storageservice 104. The ability to query the data stored at the storage service104 directly, for example, enables users to do so without having toexport the data to another data storage location such as a databasewhere the data can be queried. A user can, for example, use the storageservice 104 to query for all packets of a certain size that have aspecified destination IP address, or query for network traffic that wassent to specified geographic regions, traffic associated with aparticular protocol, traffic sent or received within a particulartimeframe, and so forth.

FIG. 3 illustrates an example of converting mirrored network traffic topacket capture data according to some embodiments. As shown in FIG. 3 ,mirrored network packets 304 are sent from computing devices associatedwith a virtual network 302 to a compute instance fleet 204, as describedin relation to FIG. 2 . FIG. 3 further illustrates the conversion of themirrored network packets 304 into packet capture data 306. As shown, thepacket capture data 306 generally can be formatted in any binary orplain-text format that represents the network packets contained in themirrored network packets 304 (for example, represented by packet capturefile 310). In some embodiments, a user can specify a type of packetcapture data 306 the user desires to use as part of the initial trafficmirroring request. In an embodiment, the processing of the mirrorednetwork packets 304 can further include the creation of metadata 308 tobe stored in association with the packet capture data. The metadata 308can be used, for example, by any other services or applications to whichthe mirrored network traffic is ultimately routed by the computeinstance fleet 204.

FIG. 4 is a diagram illustrating the use of various services of aservice provider network to process and analyze packet capture datastored at a storage service. In FIG. 4 , mirrored network traffic 404 issent from one or more virtual networks 402A-402N of a service providernetwork 100 to a compute instance fleet (not shown in FIG. 4 ) thatroutes the mirrored network traffic to a storage service 104. Asdescribed above, the compute instance fleet can convert the mirrorednetwork traffic 404 into packet capture files and store the packetcapture files at storage locations associated with various users of theservice provider network 100.

In an embodiment, the storage of the packet capture data at the storageservice 104 enables the data to be used by any of a number of otherapplications or services of a service provider network including, forexample, a threat detection service 406, a serverless compute service408, a data monitoring service 410, among other possible services. Thesevarious services can be used to perform forensic and threat analysis onthe stored, to analyze traffic patterns, to perform responsive securityor network configuration actions, among many other possible configurableactions.

FIG. 5 is a flow diagram illustrating operations 500 of a method forenabling users to mirror network traffic associated with virtualnetworks of a service provider network according to some embodiments.Some or all of the operations 500 (or other processes described herein,or variations, and/or combinations thereof) are performed under thecontrol of one or more computer systems configured with executableinstructions and are implemented as code (e.g., executable instructions,one or more computer programs, or one or more applications) executingcollectively on one or more processors, by hardware or combinationsthereof. The code is stored on a computer-readable storage medium, forexample, in the form of a computer program comprising instructionsexecutable by one or more processors. The computer-readable storagemedium is non-transitory. In some embodiments, one or more (or all) ofthe operations 500 are performed by a hardware virtualization, storageservice, and/or other components of the other figures.

The operations 500 include, at block 502, receiving a request to mirrornetwork traffic traversing a virtual network interface associated with avirtual network at the service provider network, the request identifyinga destination for the mirrored network traffic, and the virtual networkincluding a logically isolated set of computing resources associatedwith at least one user of the service provider network and hosted by atleast one computing device. In one embodiment, the virtual networkinterface is one of a group of virtual network interfaces identified inthe request to mirror the network traffic, where the group correspondsto one or more of: a set of identified virtual network interfaces, oneor more computing resources of the virtual network, a subnetwork of thevirtual network, the entire virtual network, a set of virtual networkinterfaces associated with a specified tag. Network traffic traversing avirtual network interface can include network traffic sent to and/orreceived by a virtual network interface depending on the request.

In one embodiment, the one or more operations include one or more of:adding metadata to the mirrored copy of the network traffic, convertingthe mirrored copy of the network traffic to a packet capture dataformat, and wherein the destination is one or more: a storage service, astreaming data service, a security service, an application. In oneembodiment, the destination is a storage service of the service providernetwork, and wherein the fleet of compute instances converts themirrored copy of the network traffic into one or more packet capturefiles and stores the one or more packet capture files and stores the oneor more packet capture files at one or more storage locations at thestorage service associated with at least one user of the serviceprovider network. In one embodiment, the fleet of compute instancesreceives network traffic mirrored from a plurality of virtual networksassociated with a plurality of different users of the service providernetwork.

In one embodiment, the request further specifies at least one filteridentifying selected types of network traffic to mirror, and wherein thecomputing device mirrors only network traffic matching the at least onefilter to the fleet of compute instances. In one embodiment, the requestfurther specifies one or more of: a triggering condition indicating whento begin mirroring the network traffic, and a time limit for mirroringthe network traffic.

The operations 500 further include, at block 504, identifying acomputing device hosting a computing resource to which the virtualnetwork interface is attached.

The operations 500 further include, at block 506, sending configurationinformation to the computing device, the configuration informationcausing the computing device to send a mirrored copy of network traffictraversing the virtual network interface to a fleet of compute instancesat the service provider network, the fleet of compute instances routingthe mirrored copy of network traffic to the destination.

The operations 500 further include, at block 508, receiving, by thefleet of compute instances, the mirrored copy of network traffic fromthe computing device.

The operations 500 further include, at block 510, performing one or moreoperations relative to the mirrored copy of network traffic. In oneembodiment, the one or more operations include one or more of: addingmetadata to the mirrored copy of the network traffic, converting themirrored copy of the network traffic to a packet capture data format,and wherein the destination is one or more of: a storage service, astreaming data service, a security service, an application.

The operations 500 further include, at block 512, sending the mirroredcopy of the network traffic to the destination identified by therequest.

In an embodiment, the operations further include receiving an additionalrequest to perform one or more of: stopping the mirroring of networktraffic, pausing the mirroring of network traffic, resuming themirroring of network traffic, modifying the mirroring of networktraffic. In one embodiment, the operations further include scaling thefleet of compute instances based on an amount of mirrored trafficreceived by the fleet of compute instances.

FIG. 6 illustrates an example provider network (or “service providersystem”) environment according to some embodiments. A provider network600 may provide resource virtualization to customers via one or morevirtualization services 610 that allow customers to purchase, rent, orotherwise obtain instances 612 of virtualized resources, including butnot limited to computation and storage resources, implemented on deviceswithin the provider network or networks in one or more data centers.Local Internet Protocol (IP) addresses 616 may be associated with theresource instances 612; the local IP addresses are the internal networkaddresses of the resource instances 612 on the provider network 600. Insome embodiments, the provider network 600 may also provide public IPaddresses 614 and/or public IP address ranges (e.g., Internet Protocolversion 4 (IPv4) or Internet Protocol version 6 (IPv6) addresses) thatcustomers may obtain from the provider 600.

Conventionally, the provider network 600, via the virtualizationservices 610, may allow a customer of the service provider (e.g., acustomer that operates one or more client networks 650A-650C includingone or more customer device(s) 652) to dynamically associate at leastsome public IP addresses 614 assigned or allocated to the customer withparticular resource instances 612 assigned to the customer. The providernetwork 600 may also allow the customer to remap a public IP address614, previously mapped to one virtualized computing resource instance612 allocated to the customer, to another virtualized computing resourceinstance 612 that is also allocated to the customer. Using thevirtualized computing resource instances 612 and public IP addresses 614provided by the service provider, a customer of the service providersuch as the operator of customer network(s) 650A-650C may, for example,implement customer-specific applications and present the customer'sapplications on an intermediate network 640, such as the Internet. Othernetwork entities 620 on the intermediate network 640 may then generatetraffic to a destination public IP address 614 published by the customernetwork(s) 650A-650C; the traffic is routed to the service provider datacenter, and at the data center is routed, via a network substrate, tothe local IP address 616 of the virtualized computing resource instance612 currently mapped to the destination public IP address 614.Similarly, response traffic from the virtualized computing resourceinstance 612 may be routed via the network substrate back onto theintermediate network 640 to the source entity 620.

Local IP addresses, as used herein, refer to the internal or “private”network addresses, for example, of resource instances in a providernetwork. Local IP addresses can be within address blocks reserved byInternet Engineering Task Force (IETF) Request for Comments (RFC) 1918and/or of an address format specified by IETF RFC 4193, and may bemutable within the provider network. Network traffic originating outsidethe provider network is not directly routed to local IP addresses;instead, the traffic uses public IP addresses that are mapped to thelocal IP addresses of the resource instances. The provider network mayinclude networking devices or appliances that provide network addresstranslation (NAT) or similar functionality to perform the mapping frompublic IP addresses to local IP addresses and vice versa.

Public IP addresses are Internet mutable network addresses that areassigned to resource instances, either by the service provider or by thecustomer. Traffic routed to a public IP address is translated, forexample via 1:1 NAT, and forwarded to the respective local IP address ofa resource instance.

Some public IP addresses may be assigned by the provider networkinfrastructure to particular resource instances; these public IPaddresses may be referred to as standard public IP addresses, or simplystandard IP addresses. In some embodiments, the mapping of a standard IPaddress to a local IP address of a resource instance is the defaultlaunch configuration for all resource instance types.

At least some public IP addresses may be allocated to or obtained bycustomers of the provider network 600; a customer may then assign theirallocated public IP addresses to particular resource instances allocatedto the customer. These public IP addresses may be referred to ascustomer public IP addresses, or simply customer IP addresses. Insteadof being assigned by the provider network 600 to resource instances asin the case of standard IP addresses, customer IP addresses may beassigned to resource instances by the customers, for example via an APIprovided by the service provider. Unlike standard IP addresses, customerIP addresses are allocated to customer accounts and can be remapped toother resource instances by the respective customers as necessary ordesired. A customer IP address is associated with a customer's account,not a particular resource instance, and the customer controls that IPaddress until the customer chooses to release it. Unlike conventionalstatic IP addresses, customer IP addresses allow the customer to maskresource instance or availability zone failures by remapping thecustomer's public IP addresses to any resource instance associated withthe customer's account. The customer IP addresses, for example, enable acustomer to engineer around problems with the customer's resourceinstances or software by remapping customer IP addresses to replacementresource instances.

FIG. 7 is a block diagram of an example provider network that provides astorage service and a hardware virtualization service to customers,according to some embodiments. Hardware virtualization service 720provides multiple computation resources 724 (e.g., VMs) to customers.The computation resources 724 may, for example, be rented or leased tocustomers of the provider network 700 (e.g., to a customer thatimplements customer network 750). Each computation resource 724 may beprovided with one or more local IP addresses. Provider network 700 maybe configured to route packets from the local IP addresses of thecomputation resources 724 to public Internet destinations, and frompublic Internet sources to the local IP addresses of computationresources 724.

Provider network 700 may provide a customer network 750, for examplecoupled to intermediate network 740 via local network 756, the abilityto implement virtual computing systems 792 via hardware virtualizationservice 720 coupled to intermediate network 740 and to provider network700. In some embodiments, hardware virtualization service 720 mayprovide one or more APIs 702, for example a web services interface, viawhich a customer network 750 may access functionality provided by thehardware virtualization service 720, for example via a console 794(e.g., a web-based application, standalone application, mobileapplication, etc.). In some embodiments, at the provider network 700,each virtual computing system 792 at customer network 750 may correspondto a computation resource 724 that is leased, rented, or otherwiseprovided to customer network 750.

From an instance of a virtual computing system 792 and/or anothercustomer device 790 (e.g., via console 794), the customer may access thefunctionality of storage service 710, for example via one or more APIs702, to access data from and store data to storage resources 718A-718Nof a virtual data store 716 (e.g., a folder or “bucket”, a virtualizedvolume, a database, etc.) provided by the provider network 700. In someembodiments, a virtualized data store gateway (not shown) may beprovided at the customer network 750 that may locally cache at leastsome data, for example frequently-accessed or critical data, and thatmay communicate with storage service 710 via one or more communicationschannels to upload new or modified data from a local cache so that theprimary store of data (virtualized data store 716) is maintained. Insome embodiments, a user, via a virtual computing system 792 and/or onanother customer device 790, may mount and access virtual data store 716volumes via storage service 710 acting as a storage virtualizationservice, and these volumes may appear to the user as local (virtualized)storage 798.

While not shown in FIG. 7 , the virtualization service(s) may also beaccessed from resource instances within the provider network 700 viaAPI(s) 702. For example, a customer, appliance service provider, orother entity may access a virtualization service from within arespective virtual network on the provider network 700 via an API 702 torequest allocation of one or more resource instances within the virtualnetwork or within another virtual network.

FIG. 8 illustrates an example service provider network that providesvirtual networks on the provider network to users according to someembodiments. A user's virtual network 860 at a service provider network800, for example, enables a user to connect their existinginfrastructure (for example, one or more customer devices 852) on usernetwork 850 to a set of logically isolated resource instances (forexample, VMs 824A and 824B and storage 818A and 818B), and to extendmanagement capabilities such as security services, firewalls, andintrusion detection systems to include their resource instances.

A user's virtual network 860 may be connected to a user network 850 viaa private communications channel 842. A private communications channel842 may, for example, be a tunnel implemented according to a networktunneling technology or some other technology over an intermediatenetwork 840. In some examples, the intermediate network is a sharednetwork or a public network such as the Internet. Alternatively, aprivate communications channel 842 is implemented over a direct,dedicated connection between virtual network 860 and user network 850.

A public network is broadly defined as a network that provides openaccess to and interconnectivity among a plurality of entities. Theinternet, or World Wide Web (WWW) is an example of a public network. Ashared network is broadly defined as a network to which access islimited to two or more entities, in contrast to a public network towhich access is not generally limited. A shared network, for example,can include one or more local area networks (LANs) and/or data centernetworks, or two or more LANs or data center networks that areinterconnected to form a wide area network (WAN). Examples of sharednetworks include, but are not limited to, corporate networks and otherenterprise networks. A shared network can be anywhere in scope from anetwork that covers a local area to a global network. Note that a sharednetwork can share at least some network infrastructure with a publicnetwork, and that a shared network can be coupled to one or more othernetworks, which may include a public network, with controlled accessbetween the other network(s) and the shared network. A shared networkcan also be viewed as a private network, in contrast to a public networksuch as the internet. In some embodiments, either a shared network or apublic network serves as an intermediate network between a providernetwork and a customer network.

To establish a virtual network 860 for a user on service providernetwork 800, one or more resource instances (for example, VMs 824A and824B and storage 818A and 818B) are allocated to the virtual network860. Note that other resource instances (for example, storage 818C andVMs 824C) may remain available on the service provider network 800 forother customer usage. A range of public IP addresses can also beallocated to the virtual network 860. In addition, one or morenetworking nodes (for example, routers, switches, and so forth) of theprovider network 800 can be allocated to the virtual network 860. Aprivate communications channel 842 can be established between a privategateway 862 at virtual network 860 and a gateway 856 at customer network850.

In some embodiments, in addition to, or instead of, a private gateway862, virtual network 860 includes a public gateway 864 that enablesresources within virtual network 860 to communicate directly withentities (for example, network entity 844) via intermediate network 840,and vice versa, instead of or in addition to via private communicationschannel 842.

A virtual network 860 can be optionally subdivided into two or moresubnetworks, or subnets, 870. For example, in implementations thatinclude both a private gateway 862 and a public gateway 864, a virtualnetwork 860 can be subdivided into a subnet 870A that includes resources(VMs 824A and storage 818A, in this example) reachable through privategateway 862, and a subnet 870B that includes resources (VMs 824B andstorage 818B, in this example) reachable through public gateway 864.

A user can assign allocated public IP addresses to resource instances invirtual network 860. A network entity 844 on intermediate network 840can then send traffic to a public IP address published by the user; thetraffic is routed, by the provider network 800, to the associatedresource instance. Return traffic from the resource instance is routed,by the provider network 800, back to the network entity 844 overintermediate network 840. Note that routing traffic between a resourceinstance and a network entity 844 may require network addresstranslation to translate between the public IP address and the local IPaddress of the resource instance.

Some embodiments allow users to remap public IP addresses in a user'svirtual network 860 as illustrated in FIG. 8 to devices on the user'sexternal network 850. When a packet is received (for example, fromnetwork entity 844) and the network 800 determines that the destinationIP address indicated by the packet has been remapped to an endpoint onexternal network 850, the network handles routing of the packet to therespective endpoint either via private communications channel 842 or viathe intermediate network 840. Response traffic is then routed from theendpoint to the network entity 844 through the provider network 800, oralternatively directly routed to the network entity 844 by the usernetwork 850. From the perspective of the network entity 844, it appearsas if the network entity 844 is communicating with the public IP addressof the user on the provider network 800. However, the network entity 844has actually communicated with the endpoint on customer network 850.

While FIG. 8 shows network entity 844 on intermediate network 840 andexternal to provider network 800, a network entity can be an entity onservice provider network 800. For example, one of the resource instancesprovided by provider network 800 can be a network entity that sendstraffic to a public IP address published by the customer.

FIG. 9 illustrates subnets and security groups in an example virtualnetwork 910 on a service provider network according to some embodiments.In some embodiments, a service provider network such as provider network800 in FIG. 8 allows the customer to establish and manage virtualsecurity groups 916 (for example, 916A-916D) within the user's virtualnetwork 910, within or across subnets 914. A security group 916 is alogical grouping of resource instances 918 and acts as a virtualfirewall that controls the traffic allowed to reach one or more resourceinstances 918 within the security group 916 according to security grouprules. A user can establish one or more security groups 916 within thevirtual network 910 and can associate resource instances 918 in thevirtual network 910 with one or more of the security groups 916. In someembodiments, the user establishes and/or modifies rules for eachsecurity group 916 that control the inbound traffic allowed to reach theresource instances 918 associated with the security group 916.

In the example virtual network 910 shown in FIG. 9 , the virtual network910 is subdivided into two subnets 914A and 914B. Access to the virtualnetwork 910 is controlled by gateway(s) 930. Each subnet 914 can includeat least one router 912 that acts to route traffic to (and from)resource instances 918 on the respective subnet 914. In someembodiments, network access control lists (ACLs) are used to controlaccess to the subnets 914 at router(s) 912. In the example shown in FIG.9 , resource instances 918A through 918E are on subnet 914A and resourceinstances 918F through 918J are on subnet 914B. The user has establishedfour security groups 916A through 916D. As shown in FIG. 9 , a securitygroup can extend across subnets 914, as does security group 916A thatincludes resource instances 918A and 918B on subnet 914A and resourceinstance 918F on subnet 914B. In addition, a resource instance 918 canbe included in two or more security groups 916, as is resource instance918A which is included in security group 916A and 916B.

In some embodiments, a system that implements a portion or all of thetechniques for enabling users to configure the mirroring of networktraffic sent to or received by computing resources associated with avirtual network of computing resources at a service provider network asdescribed herein may include a general-purpose computer system thatincludes or is configured to access one or more computer-accessiblemedia, such as computer system 1000 illustrated in FIG. 10 . In theillustrated embodiment, computer system 1000 includes one or moreprocessors 1010 coupled to a system memory 1020 via an input/output(I/O) interface 1030. Computer system 1000 further includes a networkinterface 1040 coupled to I/O interface 1030. While FIG. 10 showscomputer system 1000 as a single computing device, in variousembodiments a computer system 1000 may include one computing device orany number of computing devices configured to work together as a singlecomputer system 1000.

In various embodiments, computer system 1000 may be a uniprocessorsystem including one processor 1010, or a multiprocessor systemincluding several processors 1010 (e.g., two, four, eight, or anothersuitable number). Processors 1010 may be any suitable processors capableof executing instructions. For example, in various embodiments,processors 1010 may be general-purpose or embedded processorsimplementing any of a variety of instruction set architectures (ISAs),such as the x86, ARM, PowerPC, SPARC, or MIPS ISAs, or any othersuitable ISA. In multiprocessor systems, each of processors 1010 maycommonly, but not necessarily, implement the same ISA.

System memory 1020 may store instructions and data accessible byprocessor(s) 1010. In various embodiments, system memory 1020 may beimplemented using any suitable memory technology, such as random-accessmemory (RAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM),nonvolatile/Flash-type memory, or any other type of memory. In theillustrated embodiment, program instructions and data implementing oneor more desired functions, such as those methods, techniques, and datadescribed above are shown stored within system memory 1020 as code 1025and data 1026.

In one embodiment, I/O interface 1030 may be configured to coordinateI/O traffic between processor 1010, system memory 1020, and anyperipheral devices in the device, including network interface 1040 orother peripheral interfaces. In some embodiments, I/O interface 1030 mayperform any necessary protocol, timing or other data transformations toconvert data signals from one component (e.g., system memory 1020) intoa format suitable for use by another component (e.g., processor 1010).In some embodiments, I/O interface 1030 may include support for devicesattached through various types of peripheral buses, such as a variant ofthe Peripheral Component Interconnect (PCI) bus standard or theUniversal Serial Bus (USB) standard, for example. In some embodiments,the function of I/O interface 1030 may be split into two or moreseparate components, such as a north bridge and a south bridge, forexample. Also, in some embodiments some or all of the functionality ofI/O interface 1030, such as an interface to system memory 1020, may beincorporated directly into processor 1010.

Network interface 1040 may be configured to allow data to be exchangedbetween computer system 1000 and other devices 1060 attached to anetwork or networks 1050, such as other computer systems or devices asillustrated in FIG. 1 , for example. In various embodiments, networkinterface 1040 may support communication via any suitable wired orwireless general data networks, such as types of Ethernet network, forexample. Additionally, network interface 1040 may support communicationvia telecommunications/telephony networks such as analog voice networksor digital fiber communications networks, via storage area networks(SANs) such as Fibre Channel SANs, or via I/O any other suitable type ofnetwork and/or protocol.

In some embodiments, a computer system 1000 includes one or more offloadcards 1070 (including one or more processors 1075, and possiblyincluding the one or more network interfaces 1040) that are connectedusing an I/O interface 1030 (e.g., a bus implementing a version of thePeripheral Component Interconnect-Express (PCI-E) standard, or anotherinterconnect such as a QuickPath interconnect (QPI) or UltraPathinterconnect (UPI)). For example, in some embodiments the computersystem 1000 may act as a host electronic device (e.g., operating as partof a hardware virtualization service) that hosts compute instances, andthe one or more offload cards 1070 execute a virtualization manager thatcan manage compute instances that execute on the host electronic device.As an example, in some embodiments the offload card(s) 1070 can performcompute instance management operations such as pausing and/or un-pausingcompute instances, launching and/or terminating compute instances,performing memory transfer/copying operations, etc. These managementoperations may, in some embodiments, be performed by the offload card(s)1070 in coordination with a hypervisor (e.g., upon a request from ahypervisor) that is executed by the other processors 1010A-1010N of thecomputer system 1000. However, in some embodiments the virtualizationmanager implemented by the offload card(s) 1070 can accommodate requestsfrom other entities (e.g., from compute instances themselves), and maynot coordinate with (or service) any separate hypervisor.

In some embodiments, system memory 1020 may be one embodiment of acomputer-accessible medium configured to store program instructions anddata as described above. However, in other embodiments, programinstructions and/or data may be received, sent or stored upon differenttypes of computer-accessible media. Generally speaking, acomputer-accessible medium may include non-transitory storage media ormemory media such as magnetic or optical media, e.g., disk or DVD/CDcoupled to computer system 1000 via I/O interface 1030. A non-transitorycomputer-accessible storage medium may also include any volatile ornon-volatile media such as RAM (e.g., SDRAM, double data rate (DDR)SDRAM, SRAM, etc.), read only memory (ROM), etc., that may be includedin some embodiments of computer system 1000 as system memory 1020 oranother type of memory. Further, a computer-accessible medium mayinclude transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as a network and/or a wireless link, such as may be implemented vianetwork interface 1040.

In the preceding description, various embodiments are described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described.

Bracketed text and blocks with dashed borders (e.g., large dashes, smalldashes, dot-dash, and dots) are used herein to illustrate optionaloperations that add additional features to some embodiments. However,such notation should not be taken to mean that these are the onlyoptions or optional operations, and/or that blocks with solid bordersare not optional in certain embodiments.

Reference numerals with suffix letters (e.g., 718A-718N) may be used toindicate that there can be one or multiple instances of the referencedentity in various embodiments, and when there are multiple instances,each does not need to be identical but may instead share some generaltraits or act in common ways. Further, the particular suffixes used arenot meant to imply that a particular amount of the entity exists unlessspecifically indicated to the contrary. Thus, two entities using thesame or different suffix letters may or may not have the same number ofinstances in various embodiments.

References to “one embodiment,” “an embodiment,” “an exampleembodiment,” etc., indicate that the embodiment described may include aparticular feature, structure, or characteristic, but every embodimentmay not necessarily include the particular feature, structure, orcharacteristic. Moreover, such phrases are not necessarily referring tothe same embodiment. Further, when a particular feature, structure, orcharacteristic is described in connection with an embodiment, it issubmitted that it is within the knowledge of one skilled in the art toaffect such feature, structure, or characteristic in connection withother embodiments whether or not explicitly described.

Moreover, in the various embodiments described above, unlessspecifically noted otherwise, disjunctive language such as the phrase“at least one of A, B, or C” is intended to be understood to mean eitherA, B, or C, or any combination thereof (e.g., A, B, and/or C). As such,disjunctive language is not intended to, nor should it be understood to,imply that a given embodiment requires at least one of A, at least oneof B, or at least one of C to each be present.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the disclosure asset forth in the claims.

What is claimed is:
 1. A computer-implemented method comprising:receiving, by a service of a service provider network, a request tomirror network traffic traversing an existing virtual network interfaceassociated with a virtual network at the service provider network,wherein the request identifies a storage location managed by an objectstorage service of the service provider network, wherein the virtualnetwork includes a logically isolated set of computing resourcesassociated with at least one user of the service provider network, andwherein the request includes a filter identifying a type of networktraffic to mirror; sending configuration information to a computingdevice hosting a computing resource to which the existing virtualnetwork interface identified in the request is attached, wherein theconfiguration information causes the computing device to send a mirroredcopy of network traffic traversing the existing virtual networkinterface to a fleet of compute instances at the service providernetwork, and wherein the computing device mirrors network traffic basedon the filter; receiving, by the fleet of compute instances, themirrored copy of network traffic from the computing device; performingone or more operations relative to the mirrored copy of network traffic,wherein the one or more operations include converting the mirrored copyof the network traffic to a packet capture data format stored in packetcapture files; sending the packet capture files to the object storageservice for storage at the storage location identified in the request;and scaling the fleet of compute instances based on an amount ofmirrored network traffic received by the fleet of compute instances. 2.The computer-implemented method of claim 1, wherein the existing virtualnetwork interface is one of a group of virtual network interfacesidentified in the request to mirror the network traffic, the groupcorresponding to one or more of: a set of identified virtual networkinterfaces, one or more identified computing resources of the virtualnetwork, a subnetwork of the virtual network, the entire virtualnetwork, a set of virtual network interfaces associated with a specifiedtag.
 3. The computer-implemented method of claim 1, wherein the one ormore operations further includes adding metadata to the mirrored copy ofthe network traffic.
 4. A computer-implemented method comprising:receiving a request to mirror network traffic traversing an existingvirtual network interface associated with a virtual network at a serviceprovider network, wherein the request identifies a storage locationmanaged by an object storage service of the service provider network,wherein the virtual network includes a logically isolated set ofcomputing resources associated with at least one user of the serviceprovider network, and wherein the request includes a filter identifyinga type of network traffic to mirror; identifying a computing devicehosting a computing resource to which the existing virtual networkinterface identified in the request is attached; and sendingconfiguration information to the computing device hosting the computingresource to which the existing virtual network interface identified inthe request is attached, wherein the configuration information causesthe computing device to send a mirrored copy of network traffictraversing the existing virtual network interface to a fleet of computeinstances at the service provider network, wherein the computing devicemirrors the network traffic based on the filter, and wherein the fleetof compute instances causes the mirrored network traffic to be stored inone or more packet capture files at the storage location identified inthe request; and scaling the fleet of compute instances based on anamount of mirrored network traffic received by the fleet of computeinstances.
 5. The computer-implemented method of claim 4, wherein thestorage location is managed by a storage service of the service providernetwork.
 6. The computer-implemented method of claim 4, wherein theexisting virtual network interface is one of a group of virtual networkinterfaces identified in the request to mirror the network traffic, thegroup corresponding to one or more of: a set of identified virtualnetwork interfaces, one or more computing resources of the virtualnetwork, a subnetwork of the virtual network, the entire virtualnetwork, a set of virtual network interfaces associated with a specifiedtag.
 7. The computer-implemented method of claim 4, wherein the fleet ofcompute instances adds metadata to the mirrored copy of the networktraffic.
 8. The computer-implemented method of claim 4, wherein thefleet of compute instances converts the mirrored copy of the networktraffic into a packet capture data format stored in the one or morepacket capture files.
 9. The computer-implemented method of claim 4,wherein the fleet of compute instances receives network traffic mirroredfrom a plurality of virtual networks associated with a plurality ofdifferent users of the service provider network.
 10. Thecomputer-implemented method of claim 4, further comprising: receiving anadditional request to perform one or more of: stopping mirroring ofnetwork traffic at the computing device, pausing mirroring of networktraffic at the computing device, resuming mirroring of network trafficat the computing device, modifying mirroring of network traffic at thecomputing device; and sending additional configuration information tothe computing device based on the additional request.
 11. Thecomputer-implemented method of claim 4, wherein the request furtherspecifies one or more of: a triggering condition indicating when tobegin mirroring the network traffic, and a time limit for mirroring thenetwork traffic.
 12. The computer-implemented method of claim 4, furthercomprising: receiving a query including search criteria identifying asubset of data stored in the one or more packet capture files; andexecuting the query against the packet capture files stored at thestorage service.
 13. A system comprising: a hardware virtualizationservice implemented by a first one or more electronic devices, thehardware virtualization service including instructions that uponexecution cause the hardware virtualization service to: receive arequest to mirror network traffic traversing an existing virtual networkinterface associated with a virtual network at a service providernetwork, wherein the request identifies a storage location managed by anobject storage service of the service provider network, wherein thevirtual network includes a logically isolated set of computing resourcesassociated with at least one user of the service provider network andhosted by at least one computing device, and wherein the requestincludes a filter identifying a type of network traffic to mirror,identify a computing device hosting a computing resource to which theexisting virtual network interface identified in the request isattached, send configuration information to the computing device hostingthe computing resource to which the existing virtual network interfaceidentified in the request is attached, wherein the configurationinformation causes the computing device to send a mirrored copy ofnetwork traffic traversing the virtual network interface to a fleet ofcompute instances at the service provider network, and wherein thecomputing device mirrors network traffic based on the filter, and scalethe fleet of compute instances based on an amount of mirrored networktraffic received by the fleet of compute instances; and the fleet ofcompute instances implemented by a second one or more electronicdevices, the fleet of compute instances including instructions that uponexecution cause the fleet of compute instances to: receive the mirroredcopy of network traffic from the computing device, performing one ormore operations relative to the mirrored copy of network traffic,wherein the one or more operations include converting the mirrored copyof the network traffic to a packet capture data format stored in packetcapture files, and sending the packet capture files to the objectstorage service for storage at the storage location identified in therequest.
 14. The system of claim 13, wherein the existing virtualnetwork interface is one of a group of virtual network interfacesidentified in the request to mirror the network traffic, the groupcorresponding to one or more of: a set of identified virtual networkinterfaces, one or more computing resources of the virtual network, asubnetwork of the virtual network, the entire virtual network.
 15. Thesystem of claim 13, wherein the fleet of compute instances furtherinclude instructions that upon execution cause the fleet of computeinstances to add metadata to the mirrored copy of the network traffic.16. The system of claim 13, further comprising sending the packetcapture files to one or more of: a streaming data service, a securityservice, or an application.
 17. The system of claim 13, wherein thefleet of compute instances receives network traffic mirrored from aplurality of virtual networks associated with a plurality of differentusers of the service provider network.